Image coding device, image decoding device, image coding method, and image decoding method

ABSTRACT

When a coding mode selected by an encoding controlling part 1 is an intra prediction mode, an intra prediction part 4 carries out an intra-frame prediction process using pixels adjacent to a partition Pin which is generated through a division by a block dividing part 2 or pixels adjacent to a higher layer partition Pin−1 to which the partition Pin belongs to generate a prediction image (Pin).

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a Divisional of copending U.S. application Ser. No.16/545,740, filed on Aug. 20, 2019, which is a Divisional of U.S.application Ser. No. 13/991,041, filed on Jul. 2, 2013 (now U.S. Pat.No. 10,432,956 issued on Oct. 1, 2019), which is the National Phaseunder 35 U.S.C. § 371 of International Application No.PCT/JP2011/006116, filed on Nov. 1, 2011, which claims the benefit under35 U.S.C. § 119(a) to Patent Application No. 2010-281743, filed in Japanon Dec. 17, 2010, all of which are hereby expressly incorporated byreference into the present application.

FIELD OF THE INVENTION

The present invention relates to an image coding device for and an imagecoding method of encoding a moving image with a high degree ofefficiency, and an image decoding device for and an image decodingmethod of decoding an encoded moving image with a high degree ofefficiency.

BACKBROUND OF THE INVENTION

For example, in an intra prediction mode for luminance in an AVC/H.264(ISO/IEC 14496-10|ITU-T H.264) encoding method which is an internationalstandard method, one prediction mode can be selected from among aplurality of prediction modes for each block (for example, refer tononpatent reference 1). FIG. 14 is an explanatory drawing showing intraprediction modes in the case of a 4×4 pixel block size for luminance. Inthe case of a 4×4 pixel block size for luminance, nine intra predictionmodes (a mode 0, a mode 1, . . . , and a mode 8) are defined.

In the example shown in FIG. 14, each white circle shows a pixel in ablock to be encoded. Each black circle shows a pixel that is used forprediction, and that exists in an already-encoded adjacent block. Themode 2 is the one in which an average prediction is carried out in sucha way that each pixel in the block to be encoded is predicted by usingthe average of adjacent pixels existing in the upper and left blocks.Each of the modes other than the mode 2 is the one in which adirectional prediction is carried out. Particularly, the mode 0 is theone in which a vertical prediction is carried out in such a way thatadjacent pixels in the upper block are repeatedly replicated to createplural rows of pixels along a vertical direction to generate aprediction image. For example, the mode 0 is selected when the block tobe encoded is a vertically striped pattern. The mode 1 is the one inwhich a horizontal prediction is carried out in such a way that adjacentpixels in the left block are repeatedly replicated to create pluralcolumns of pixels along a horizontal direction to generate a predictionimage. Further, in each of the modes 3 to 8, interpolation pixelsrunning in a predetermined direction (i.e., a direction shown by arrows)are generated by using the adjacent pixels in the upper block or theleft block to generate a prediction image.

The block size for luminance to which an intra prediction is applied canbe selected from 4×4 pixels, 8×8 pixels, and 16×16 pixels. In the casein which the block size for luminance is 8×8 pixels, nine intraprediction modes are defined, like in the case in which the block sizefor luminance is 4×4 pixels. In contrast with this, in the case in whichthe block size for luminance is 16×16 pixels, four intra predictionmodes which are called plane predictions are defined in addition to anaverage prediction mode, a vertical prediction mode, and a horizontalprediction mode. A plane prediction mode is a mode in which pixelscreated by carrying out an interpolation in a diagonal direction on theadjacent pixels in the upper block and the adjacent pixels in the leftblock are provided as predicted values.

Although in a directional prediction mode in the case in which the blocksize for luminance is 4×4 pixels or 8×8 pixels, because predicted valuesare generated along a direction predetermined according to the mode(e.g., a direction of 45 degrees), the prediction efficiency increasesand the code amount can be reduced when the direction of a boundary(edge) of an object in the block matches the direction shown by theprediction mode, the prediction efficiency decreases when the directionof an edge of an object in the block does not match the direction shownby the prediction mode. In contrast with this, because an increase inthe number of selectable directional prediction modes causes an increasein the probability that the direction of an edge matches the directionshown by a prediction mode, it can be assumed that the predictionefficiency increases.

RELATED ART DOCUMENT Nonpatent Reference

Nonpatent reference 1: MPEG-4 AVC (ISO/IEC 14496-10)/H.ITU-T 264standards

SUMMARY OF THE INVENTION Problems to be Solved by the Invention

Because the conventional image coding device is constructed as above, anincrease in the number of selectable directional prediction modesincreases the probability that the direction of an edge matches thedirection shown by a prediction mode, and hence the predictionefficiency can be improved. A problem is, however, that because thenumber of choices of similar prediction images increases even if thenumber of selectable directional predictions is increased, animprovement in the encoding efficiency commensurate with an increase inthe amount of computation which is caused by the increase in the numberof selectable directional predictions is not provided, and the increasein the number of selectable directional prediction has a limitedcontribution to improving the encoding efficiency.

The present invention is made in order to solve the above-mentionedproblem, and it is therefore an object of the present invention toprovide an image coding device and an image coding method capable ofincreasing the degree of improvement in the encoding efficiency which iscaused by an increase in the number of directional prediction modes witha small amount of computation and a small code amount. It is anotherobject of the present invention to provide an image decoding device andan image decoding method capable of correctly decoding coded data inwhich an improvement in the encoding efficiency is provided to acquire amoving image.

Means for Solving the Problem

In accordance with the present invention, there is provided an imagecoding device in which when a coding mode selected by an encodingcontrolling unit is an intra prediction mode, a prediction imagegenerating unit carries out an intra-frame prediction process usingpixels adjacent to each coding block which is generated through adivision by a block dividing unit or pixels adjacent to a higher layercoding block to which each coding block belongs to generate a predictionimage. Further, when the coding mode selected by the encodingcontrolling unit is an intra prediction mode, a variable length encodingunit encodes a flag indicating whether an intra prediction direction ofa coding block adjacent to the coding block which is a target to beprocessed and which is generated through the division by the blockdividing unit matches that of the target coding block, and, when theflag indicates that the intra prediction direction of an adjacent codingblock matches that of the coding block, also encodes information showingwhether which one of adjacent coding blocks has an intra predictiondirection matching that of the coding block.

Advantages of the Invention

Because the image coding device in accordance with the present inventionis constructed in such a way that when the coding mode selected by theencoding controlling unit is an intra prediction mode, the predictionimage generating unit carries out an intra-frame prediction processusing pixels adjacent to each coding block which is generated throughthe division by the block dividing unit or pixels adjacent to a higherlayer coding block to which each coding block belongs to generate aprediction image, there is provided an advantage of being able toincrease the degree of improvement in the encoding efficiency which iscaused by an increase in the number of directional predictions with asmall amount of computation and a small code amount.

BRIEF DESCRIPTION OF THE FIGURES

FIG. 1 is a block diagram showing an image coding device in accordancewith Embodiment 1 of the present invention;

FIG. 2 is a flow chart showing processing carried out by the imagecoding device in accordance with Embodiment 1 of the present invention;

FIG. 3 is an explanatory drawing showing a state in which each codingblock having a maximum size is hierarchically divided into a pluralityof coding blocks;

FIG. 4(a) is an explanatory drawing showing a distribution of partitionsinto which a block to encoded is divided, and FIG. 4(b) is anexplanatory drawing showing a state in which a coding mode m (B^(n)) isassigned to each of the partitions after a hierarchical layer divisionis performed by using a quadtree graph;

FIG. 5 is an explanatory drawing showing an example of an intraprediction parameter (intra prediction mode) which can be selected foreach partition P_(i) ^(n) belonging to a coding block B^(n);

FIG. 6 is an explanatory drawing showing an example of pixels which areused when generating a predicted value of each pixel in a partitionP_(i) ^(n) in the case of l_(i) ^(n)=m_(i) ^(n)=4;

FIG. 7 is an explanatory drawing showing a relationship between apartition P_(i) ^(n) and a higher layer partition P_(i) ^(n−1);

FIG. 8 is an explanatory drawing showing a partition P_(i) ^(n)belonging to a coding block B^(n);

FIG. 9 is an explanatory drawing showing an example of an intraprediction parameter (intra prediction mode) which can be selected foreach partition P_(i) ^(n) belonging to a coding block B^(n);

FIG. 10 is a block diagram showing an image decoding device inaccordance with Embodiment 1 of the present invention;

FIG. 11 is a flow chart showing processing carried out by the imagedecoding device in accordance with Embodiment 1 of the presentinvention;

FIG. 12 is an explanatory drawing showing coded data of each partition;

FIG. 13 is an explanatory drawing showing an adjacent partition; and

FIG. 14 is an explanatory drawing showing intra prediction modes in thecase of a 4×4 pixel block size for luminance.

EMBODIMENTS OF THE INVENTION

Hereafter, in order to explain this invention in greater detail, thepreferred embodiments of the present invention will be described withreference to the accompanying drawings.

Embodiment 1

FIG. 1 is a block diagram showing a moving image encoding device inaccordance with Embodiment 1 of the present invention. Referring to FIG.1, an encoding controlling part 1 carries out a process of determining amaximum size of each of coding blocks which is a unit to be processed ata time when an intra prediction process (intra-frame prediction process)or a motion-compensated prediction process (inter-frame predictionprocess) is carried out, and also determining an upper limit on thenumber of hierarchical layers in a hierarchy in which each of the codingblocks having the maximum size is hierarchically divided into blocks.The encoding controlling part 1 also carries out a process of selectinga coding mode suitable for each of the coding blocks into which eachcoding block having the maximum size is divided hierarchically from oneor more available coding modes (one or more intra coding modes and oneor more inter coding modes). The encoding controlling part 1 furthercarries out a process of determining a quantization parameter and atransformation block size which are used when a difference image iscompressed for each coding block, and also determining an intraprediction parameter or an inter prediction parameter which is used whena prediction process is carried out for each coding block. Thequantization parameter and the transformation block size are included inprediction difference encoding parameters, and these prediction errorencoding parameters are outputted to a transformation/quantization part7, an inverse quantization/inverse transformation part 8, a variablelength encoding part 13, and so on. The encoding controlling part 1constructs an encoding controlling unit.

A block dividing part 2 carries out a process of, when receiving a videosignal showing an inputted image (current picture), dividing theinputted image into coding blocks each having the maximum sizedetermined by the encoding controlling part 1, and also dividing each ofthe coding blocks into blocks hierarchically until the number ofhierarchical layers reaches the upper limit on the number ofhierarchical layers which is determined by the encoding controlling part1. The block dividing part 2 constructs a block dividing unit. Aselection switch 3 carries out a process of, when the coding modeselected by the encoding controlling part 1 for a coding block, which isgenerated through the division by the block dividing part 2, is an intracoding mode, outputting the coding block to an intra prediction part 4,and, when the coding mode selected by the encoding controlling part 1for the coding block, which is generated through the division by theblock dividing part 2, is an inter coding mode, outputting the codingblock to a motion-compensated prediction part 5.

The intra prediction part 4 carries out a process of, when receiving thecoding block, which is generated through the division by the blockdividing part 2, from the selection switch 3, carrying out an intraprediction process based on the intra prediction parameter outputtedthereto from the encoding controlling part on the coding block togenerate a prediction image by using already-encoded pixels which arestored in a memory 10 for intra prediction and which are adjacent to thecoding block or already-encoded pixels which are adjacent to a higherlayer coding block to which the coding block belongs. Themotion-compensated prediction part 5 carries out a process of, when aninter coding mode is selected by the encoding controlling part 1 as acoding mode corresponding to the coding block, which is generatedthrough the division by the block dividing part 2, performing amotion-compensated prediction process on the coding block to generate aprediction image by using one or more frames of reference images storedin a motion-compensated prediction frame memory 12 on the basis of theinter prediction parameter outputted thereto from the encodingcontrolling part 1. A prediction image generating unit is comprised ofthe selection switch 3, the intra prediction part 4, and themotion-compensated prediction part 5.

A subtracting part 6 carries out a process of subtracting the predictionimage generated by the intra prediction part 4 or the motion-compensatedprediction part 5 from the coding block, which is generated through thedivision by the block dividing part 2, to generate a difference image(=the coding block−the prediction image). The subtracting part 6constructs a difference image generating unit. Thetransformation/quantization part 7 carries out a process of performingan orthogonal transformation process (e.g., a DCT (discrete cosinetransform) or an orthogonal transformation process, such as a KLtransform, in which bases are designed for a specific learning sequencein advance) on the difference image generated by the subtracting part 6in units of a block having a transformation block size included in theprediction difference encoding parameters outputted from the encodingcontrolling part 1, and also quantizing the transform coefficients ofthe difference image by using a quantization parameter included in theprediction difference encoding parameters to output the transformcoefficients quantized thereby as compressed data of the differenceimage. The transformation/quantization part 7 constructs an imagecompression unit.

The inverse quantization/inverse transformation part 8 carries out aprocess of inverse-quantizing the compressed data outputted thereto fromthe transformation/quantization part 7 by using the quantizationparameter included in the prediction difference encoding parametersoutputted from the encoding controlling part 1, and performing aninverse transformation process (e.g., an inverse DCT (inverse discretecosine transform) or an inverse transformation process such as aninverse KL transform) on the compressed data inverse-quantized therebyto output the compressed data on which the inverse quantization/inversetransformation part has carried out the inverse transformation processas a local decoded prediction difference signal.

An adding part 9 carries out a process of adding the local decodedprediction difference signal outputted thereto from the inversequantization/inverse transformation part 8 and the prediction signalshowing the prediction image generated by the intra prediction part 4 orthe motion-compensated prediction part 5 to generate a local decodedimage signal showing a local decoded image. The memory 10 for intraprediction is a recording medium, such as a RAM, for storing the localdecoded image shown by the local decoded image signal generated by theadding part 9 as an image which the intra prediction part 4 will usewhen performing the intra prediction process the next time.

A loop filter part 11 carries out a process of compensating for anencoding distortion included in the local decoded image signal generatedby the adding part 9, and outputting the local decoded image shown bythe local decoded image signal on which the loop filter part has carriedout the encoding distortion compensation to the motion-compensatedprediction frame memory 12 as a reference image. The motion-compensatedprediction frame memory 12 is a recording medium, such as a RAM, forstoring the local decoded image on which the loop filter part 11 hascarried out the filtering process as a reference image which themotion-compensated prediction part 5 will use when performing themotion-compensated prediction process the next time.

The variable length encoding part 13 carries out a process ofvariable-length-encoding the compressed data outputted thereto from thetransformation/quantization part 7, the coding mode and the predictiondifference encoding parameters which are outputted thereto from theencoding controlling part 1, and the intra prediction parameteroutputted thereto from the intra prediction part 4 or the interprediction parameter outputted thereto from the motion-compensatedprediction part 5 to generate a bitstream into which coded data of thecompressed data, coded data of the coding mode, coded data of theprediction difference encoding parameters, and coded data of the intraprediction parameter or the inter prediction parameter are multiplexed.The variable length encoding part 13 constructs a variable lengthencoding unit.

In the example of FIG. 1, the encoding controlling part 1, the blockdividing part 2, the selection switch 3, the intra prediction part 4,the motion-compensated prediction part 5, the subtracting part 6, thetransformation/quantization part 7, the inverse quantization/inversetransformation part 8, the adding part 9, the loop filter part 11, andthe variable length encoding part 13, which are the components of themoving image encoding device, can consist of pieces of hardware forexclusive use (e.g., integrated circuits in each of which a CPU ismounted, one chip microcomputers, or the like), respectively. As analternative, the moving image encoding device can consist of a computer,and part or all of a program in which the processes carried out by theencoding controlling part 1, the block dividing part 2, the selectionswitch 3, the intra prediction part 4, the motion-compensated predictionpart 5, the subtracting part 6, the transformation/quantization part 7,the inverse quantization/inverse transformation part 8, the adding part9, the loop filter part 11, and the variable length encoding part 13 aredescribed can be stored in a memory of the computer and the CPU of thecomputer can be made to execute the program stored in the memory. FIG. 2is a flow chart showing the processing carried out by the moving imageencoding device in accordance with Embodiment 1 of the presentinvention.

FIG. 10 is a block diagram showing a moving image decoding device inaccordance with Embodiment 1 of the present invention. Referring to FIG.10, a variable length decoding part 21 carries out a process ofspecifying the maximum size of each coding block which is a unit to beprocessed at a time when an intra prediction process or amotion-compensated prediction process is carried out, and the number ofhierarchical layers in the hierarchy in which each of the coding blockshaving the maximum size is hierarchically divided into blocks to specifythe coded data associated with each of the coding blocks having themaximum size and the coded data associated with each of the codingblocks into which each of the coding blocks having the maximum size isdivided hierarchically from the coded data multiplexed into thebitstream, and variable-length-decoding the coded data associated witheach of the coding blocks to acquire the compressed data, the codingmode, the prediction difference encoding parameters, and the intraprediction parameter or the inter prediction parameter, which areassociated with each of the coding blocks, and outputting the compresseddata and the prediction difference encoding parameters to an inversequantization/inverse transformation part 25, and also outputting thecoding mode, and the intra prediction parameter or the inter predictionparameter to a selection switch 22. The variable length decoding part 21constructs a variable length decoding unit.

The selection switch 22 carries out a process of, when the coding modeassociated with the coding block, which is outputted from the variablelength decoding part 21, is an intra coding mode, outputting the intraprediction parameter outputted thereto from the variable length decodingpart 21 to an intra prediction part 23, and, when the coding mode is aninter coding mode, outputting the inter prediction parameter outputtedthereto from the variable length decoding part 21 to a motioncompensation part 24.

The intra prediction part 23 carries out a process of performing anintra-frame prediction process on the coding block to generate aprediction image by using either already-decoded pixels which are storedin a memory 27 for intra prediction and which are adjacent to the codingblock or already-decoded pixels which are adjacent to a higher layercoding block to which the coding block belongs on the basis of the intraprediction parameter outputted thereto from the selection switch 22. Themotion compensation part 24 carries out a process of performing amotion-compensated prediction process on the coding block to generate aprediction image by using one or more frames of reference images storedin a motion-compensated prediction frame memory 29 on the basis of theinter prediction parameter outputted thereto from the selection switch22. A prediction image generating unit is comprised of the selectionswitch 22, the intra prediction part 23, and the motion compensationpart 24.

The inverse quantization/inverse transformation part 25 carries out aprocess of inverse-quantizing the compressed data associated with thecoding block, which are outputted thereto from the variable lengthdecoding part 21, by using the quantization parameter included in theprediction difference encoding parameters outputted thereto from thevariable length decoding part 21, and performing an inversetransformation process (e.g., an inverse DCT (inverse discrete cosinetransform) or an inverse transformation process such as an inverse KLtransform) on the compressed data inverse-quantized thereby in units ofa block having the transformation block size included in the predictiondifference encoding parameters, and outputting the compressed data onwhich the inverse quantization/inverse transformation part has carriedout the inverse transformation process as a decoded predictiondifference signal (signal showing a pre-compressed difference image).The inverse quantization/inverse transformation part 26 constructs adifference image generating unit.

An adding part 26 carries out a process of adding the decoded predictiondifference signal outputted thereto from the inversequantization/inverse transformation part 25 and the prediction signalshowing the prediction image generated by the intra prediction part 23or the motion compensation part 24 to generate a decoded image signalshowing a decoded image. The adding part 26 constructs a decoded imagegenerating unit. The memory 27 for intra prediction is a recordingmedium, such as a RAM, for storing the decoded image shown by thedecoded image signal generated by the adding part 26 as an image whichthe intra prediction part 23 will use when performing the intraprediction process the next time.

A loop filter part 58 carries out a process of compensating for anencoding distortion included in the decoded image signal generated bythe adding part 26, and outputting the decoded image shown by thedecoded image signal on which the loop filter part has carried out theencoding distortion compensation to the motion-compensated predictionframe memory 29 as a reference image and also outputting the decodedimage to outside the moving image decoding device as a reproduced image.The motion-compensated prediction frame memory 29 is a recording medium,such as a RAM, for storing the decoded image on which the loop filterpart 28 has carried out the filtering process as a reference image whichthe motion compensation part 24 will use when performing themotion-compensated prediction process the next time.

In the example of FIG. 10, the variable length decoding part 21, theselection switch 22, the intra prediction part 23, the motioncompensation part 24, the inverse quantization/inverse transformationpart 25, the adding part 26, and the loop filter part 28, which are thecomponents of the moving image decoding device, can consist of pieces ofhardware for exclusive use (e.g., integrated circuits in each of which aCPU is mounted, one chip microcomputers, or the like), respectively. Asan alternative, the moving image decoding device can consist of acomputer, and part or all of a program in which the processes carriedout by the variable length decoding part 21, the selection switch 22,the intra prediction part 23, the motion compensation part 24, theinverse quantization/inverse transformation part 25, the adding part 26,and the loop filter part 28 are described can be stored in a memory ofthe computer and the CPU of the computer can be made to execute theprogram stored in the memory. FIG. 11 is a flow chart showing theprocessing carried out by the moving image decoding device in accordancewith Embodiment 1 of the present invention.

The moving image encoding device in accordance with this Embodiment 1 ischaracterized in that the moving image encoding device adapts itself toa local change of a video signal in space and time directions to dividethe video signal into regions of various sizes, and carries outintra-frame and inter-frame adaptive encoding. In general, a videosignal has a characteristic of its complexity varying locally in spaceand time. There can be a case in which a pattern having a uniform signalcharacteristic in a relatively large image area, such as a sky image ora wall image, or a pattern having a complicated texture pattern in asmall image area, such as a person image or a picture including a finetexture, also coexists on a certain video frame from the viewpoint ofspace.

Also from the viewpoint of time, a relatively large image area, such asa sky image or a wall image, has a small local change in a timedirection in its pattern, while an image of a moving person or objecthas a larger temporal change because its outline has a movement of arigid body and a movement of a non-rigid body with respect to time.Although a process of generating a prediction difference signal havingsmall signal power and small entropy by using temporal and spatialprediction, thereby reducing the whole code amount, is carried out inthe encoding process, the code amount of parameters used for theprediction can be reduced as long as the parameters can be applieduniformly to as large an image signal region as possible. On the otherhand, because the amount of errors occurring in the prediction increaseswhen the same prediction parameter is applied to a large image area inan image signal pattern having a large change in time and space, thecode amount of the prediction difference signal cannot be reduced.Therefore, it is desirable to reduce the size of a region which issubjected to the prediction process when performing the predictionprocess on an image area having a large change in time and space,thereby reducing the electric power and entropy of the predictiondifference signal, even though the data volume of the predictionparameter which is used for the prediction process is increased. Inorder to carry out an encoding process which is adapted for such thetypical characteristics of a video signal, the moving image encodingdevice in accordance with this Embodiment 1 adopts a structure ofhierarchically dividing each region having a predetermined maximum blocksize of the video signal into blocks, and adapting the predictionprocess and the encoding process of encoding a prediction difference foreach of the blocks into which each region is divided.

A video signal which is to be processed by the moving image encodingdevice in accordance with this Embodiment 1 can be an arbitrary videosignal in which each video frame consists of a series of digital samples(pixels) in two dimensions, horizontal and vertical, such as a YUVsignal which consists of a luminance signal and two color differencesignals, a color video image signal in arbitrary color space, such as anRGB signal, outputted from a digital image sensor, a monochrome imagesignal, or an infrared image signal. The gradation of each pixel can bean 8-bit, 10-bit, or 12-bit one. In the following explanation, theinputted video signal is a YUV signal unless otherwise specified. It isfurther assumed that the two color difference components U and V aresignals having a 4:2:0 format which are subsampled with respect to theluminance component Y. A data unit to be processed which corresponds toeach frame of the video signal is referred to as a “picture.” In thisEmbodiment 1, a “picture” is explained as a video frame signal on whichprogressive scanning has been carried out. When the video signal is aninterlaced signal, a “picture” can be alternatively a field image signalwhich is a unit which constructs a video frame.

Next, the operation of the moving image encoding device and theoperation of the moving image decoding device will be explained.

First, the processing carried out by the moving image encoding deviceshown in FIG. 1 will be explained. First, the encoding controlling part1 determines a maximum size of each of coding blocks which is a unit tobe processed at a time when an intra prediction process (intra-frameprediction process) or a motion-compensated prediction process(inter-frame prediction process) is carried out, and also determines anupper limit on the number of hierarchical layers in a hierarchy in whicheach of the coding blocks having the maximum size is hierarchicallydivided into blocks (step ST1 of FIG. 2).

As a method of determining the maximum size of each of coding blocks,for example, there is considered a method of determining a maximum sizefor all the pictures according to the resolution of the inputted image.Further, there can be considered a method of quantifying a variation inthe complexity of a local movement of the inputted image as a parameterand then determining a small size for a picture having a large andvigorous movement while determining a large size for a picture having asmall movement. As a method of determining the upper limit on the numberof hierarchical layers, for example, there can be considered a method ofincreasing the depth of the hierarchy, i.e., the number of hierarchicallayers to make it possible to detect a finer movement as the inputtedimage has a larger and more vigorous movement, or decreasing the depthof the hierarchy, i.e., the number of hierarchical layers as theinputted image has a smaller movement.

The encoding controlling part 1 also selects a coding mode correspondingto each of the coding blocks into which each coding block having themaximum size is divided hierarchically from one or more available codingmodes (M intra coding modes and N inter coding modes) (step ST2). The Mintra coding modes which are prepared in advance will be mentionedbelow. When each of the coding block into which the inputted image ishierarchically divided by the block dividing unit 2 which will bementioned below is further divided into partitions, the encodingcontrolling part can select a coding mode corresponding to each of thepartitions. Hereafter, an explanation will be made in this Embodiment 1by assuming that each of the coding blocks is further divided intopartitions. Although a detailed explanation of a selection method ofselecting a coding mode for use in the encoding controlling part 1 willbe omitted because the selection method is a known technique, there is amethod of carrying out an encoding process on each coding block by usingan arbitrary available coding mode to examine the encoding efficiencyand select a coding mode having the highest level of encoding efficiencyfrom among a plurality of available coding modes, for example.

The encoding controlling part 1 further determines a quantizationparameter and a transformation block size which are used when adifference image is compressed for each partition belonging to eachcoding block, and also determines an intra prediction parameter or aninter prediction parameter which is used when a prediction process iscarried out. The encoding controlling part 1 outputs predictiondifference encoding parameters including the quantization parameter andthe transformation block size to the transformation/quantization part 7,the inverse quantization/inverse transformation part 8, and the variablelength encoding part 13. The encoding controlling part also outputs theprediction difference encoding parameters to the intra prediction part 4as needed.

When receiving the video signal showing the inputted image, the blockdividing part 2 divides the inputted image into coding blocks eachhaving the maximum size determined by the encoding controlling part 1,and also divides each of the coding blocks into blocks hierarchicallyuntil the number of hierarchical layers reaches the upper limit on thenumber of hierarchical layers which is determined by the encodingcontrolling part 1. The block dividing part further divides each of thecoding blocks into partitions (step ST3). FIG. 3 is an explanatorydrawing showing a state in which each coding block having the maximumsize is hierarchically divided into a plurality of coding blocks. In theexample of FIG. 3, each coding block having the maximum size is a codingblock B⁰ in the 0th hierarchical layer, and its luminance component hasa size of (L⁰, M⁰). Further, in the example of FIG. 3, by carrying outthe hierarchical division with this coding block B⁰ having the maximumsize being set as a starting point until the depth of the hierarchyreaches a predetermined depth which is set separately according to aquadtree structure, coding blocks B^(n) can be acquired.

At the depth of n, each coding block B^(n) is an image area having asize of (L^(n), M^(n)). Although L^(n) can be the same as or differ fromMn, the case of L^(n)=M^(n) is shown in the example of FIG. 3.Hereafter, the size of each coding block B^(n) is defined as the size of(L^(n), M^(n)) in the luminance component of the coding block B^(n).

Because the block dividing part 2 carries out a quadtree division,(L^(n+1), M^(n+1))=(L^(n)/2, M^(n)/2) is always established. In the caseof a color video image signal (4:4:4 format) in which all the colorcomponents have the same sample number, such as an RGB signal, all thecolor components have a size of (L^(n), M^(n)), while in the case ofhandling a 4:2:0 format, a corresponding color difference component hasan encoding block size of (L^(n)/2, M^(n)/2). Hereafter, a coding modeselectable for each coding block B^(n) in the nth hierarchical layer isexpressed as m (B^(n)).

In the case of a color video signal which consists of a plurality ofcolor components, the coding mode m (B^(n)) can be formed in such a waythat an individual mode is used for each color component. Hereafter, anexplanation will be made by assuming that the coding mode m (B^(n))indicates the one for the luminance component of each coding blockhaving a 4:2:0 format in a YUV signal unless otherwise specified. Thecoding mode m (B^(n)) can be one of one or more intra coding modes(generically referred to as “INTRA”) or one or more inter coding modes(generically referred to as “INTER”), and the encoding controlling part1 selects, as the coding mode m (B^(n)), a coding mode with the highestdegree of encoding efficiency for each coding block B^(n) from among allthe coding modes available in the picture currently being processed or asubset of these coding modes, as mentioned above.

Each coding block B^(n) is further divided into one or more predictionunits (partitions) by the block dividing part, as shown in FIG. 3.Hereafter, each partition belonging to each coding block B^(n) isexpressed as P_(i) ^(n) (i shows a partition number in the nthhierarchical layer). FIG. 8 is an explanatory drawing showing apartition P_(i) ^(n) belonging to a coding block B^(n). How the divisionof each coding block B^(n) into partitions P_(i) ^(n) belonging to thecoding block B^(n) is carried out is included as information in thecoding mode m (B^(n)). While the prediction process is carried out oneach of all the partitions P_(i) ^(n) according to the coding mode m(B^(n)), an individual prediction parameter can be selected for eachpartition P_(i) ^(n).

The encoding controlling part 1 produces such a block division state asshown in, for example, FIG. 4 for a coding block having the maximumsize, and then determines coding blocks B^(n). Hatched shown in FIG.4(a) show a distribution of partitions into which the coding blockhaving the maximum size is divided, and FIG. 4(b) shows a situation inwhich coding modes m (B^(n)) are respectively assigned to the partitionsgenerated through the hierarchical layer division by using a quadtreegraph. Each node enclosed by □ shown in FIG. 4(b) is a node (codingblock B^(n)) to which a coding mode m (B^(n)) is assigned.

When the encoding controlling part 1 selects an intra coding mode (m(B^(n)) ∈ INTRA), the selection switch 3 outputs each partition P_(i)^(n) belonging to the coding block B^(n) which is generated through thedivision by the block dividing unit 2 to the intra prediction part 4,whereas when the encoding controlling part 1 selects an inter codingmode (m (B^(n)) ∈ INTER), the selection switch 3 outputs each partitionP_(i) ^(n) belonging to the coding block B^(n) to the motion-compensatedprediction part 5.

When receiving each partition P_(i) ^(n) belonging to the coding blockB^(n) from the selection switch 3 (step ST4), the intra prediction part4 carries out an intra prediction process on each partition P_(i) ^(n)to generate an intra prediction image (P_(i) ^(n)) on the basis of theintra prediction parameter determined by the encoding controlling part 1(step ST5), as will be concretely described below. Hereafter, in thisspecification, P_(i) ^(n) denotes a partition and (P_(i) ^(n)) denotes aprediction image of the partition P_(i) ^(n).

Because the moving image decoding device also needs to generate an intraprediction image (P_(i) ^(n)) which is completely the same as the intraprediction image, the intra prediction parameter used for the generationof the intra prediction image (P_(i) ^(n)) are multiplexed into thebitstream by the variable length encoding part 13. The number of intraprediction directions each of which can be selected as an intraprediction parameter can be configured in such a way as to differaccording to the size of the target block to be processed. The number ofintra prediction directions selectable for a large-size partition can bedecreased because the efficiency of intra prediction decreases, whilethe number of intra prediction directions selectable for a small-sizepartition can be increased. For example, 34 intra prediction directionscan be provided for a 4×4-pixel partition and an 8×8-pixel partition, 17intra prediction directions can be provided for a 16×16-pixel partition,and 9 intra prediction directions can be provided for a 32×32-pixelpartition.

When receiving each partition P_(i) ^(n) belonging to the coding blockB^(n) from the selection switch 3 (step ST4), the motion-compensatedprediction part 5 carries out an inter prediction process on eachpartition P_(i) ^(n) to generate an inter prediction image (P_(i) ^(n))on the basis of the inter prediction parameter determined by theencoding controlling part 1 (step ST6). More specifically, themotion-compensated prediction part 5 carries out a motion-compensatedprediction process on each partition P_(i) ^(n) to generate an interprediction image (P_(i) ^(n)) by using one or more frames of referenceimages stored in the motion-compensated prediction frame memory 12 onthe basis of the inter prediction parameter outputted thereto from theencoding controlling part 1. Because the moving image decoding devicealso needs to generate an inter prediction image (P_(i) ^(n)) which iscompletely the same as the inter prediction image, the inter predictionparameter used for the generation of the inter prediction image (P_(i)^(n)) are multiplexed into the bitstream by the variable length encodingpart 13.

When receiving the prediction image (P_(i) ^(n)) from the intraprediction part 4 or the motion-compensated prediction part 5, thesubtracting part 6 subtracts the prediction image (P_(i) ^(n)) from thepartition P_(i) ^(n) belonging to the coding block B^(n) which isgenerated through the division by the block dividing unit 2 to generatea prediction difference signal e_(i) ^(n) showing a difference image(step ST7). When the subtracting part 6 generates the predictiondifference signal e_(i) ^(n), the transformation/quantization part 7carries out a transformation process (e.g., a DCT (discrete cosinetransform) or an orthogonal transformation process, such as a KLtransform, in which bases are designed for a specific learning sequencein advance) on the prediction difference signal e_(i) ^(n) in units of ablock having the transformation block size included in the predictiondifference encoding parameters outputted thereto from the encodingcontrolling part 1, and also quantizes the transform coefficients of theprediction difference signal e_(i) ^(n) by using the quantizationparameter included in the prediction difference encoding parameters andoutputs compressed data of the difference image which are the transformcoefficients quantized thereby to the inverse quantization/inversetransformation part 8 and the variable length encoding part 13 (stepST8).

When receiving the compressed data of the difference image from thetransformation/quantization part 7, the inverse quantization/inversetransformation part 8 inverse-quantizes the compressed data of thedifference image by using the quantization parameter included in theprediction difference encoding parameters outputted thereto from theencoding controlling part 1, performs an inverse transformation process(e.g., an inverse DCT (inverse discrete cosine transform) or an inversetransformation process such as an inverse KL transform) on thecompressed data inverse-quantized thereby in units of a block having thetransformation block size included in the prediction difference encodingparameters, and outputs the compressed data on which the inversequantization/inverse transformation part has carried out the inversetransformation process to the adding part 9 as a local decodedprediction difference signal (step ST9).

When receiving the local decoded prediction difference signal from theinverse quantization/inverse transformation part 8, the adding part 9adds the local decoded prediction difference signal and the predictionsignal showing the prediction image (P_(i) ^(n)) generated by the intraprediction part 4 or the motion-compensated prediction part 5 togenerate a local decoded image signal showing a local decoded partitionimage or a local decoded coding block image (referred to as a localdecoded image from here on) which is a group of local decoded partitionimages, and outputs the local decoded image signal to the loop filterunit 11 (step ST10). Further, this local decoded image is stored in thememory 10 for intra prediction in order that it can be used for intraprediction.

When receiving the local decoded image signal from the adding part 9,the loop filter part 11 compensates for an encoding distortion includedin the local decoded image signal, and stores the local decoded imageshown by the local decoded image signal on which the loop filter parthas carried out the encoding distortion compensation in themotion-compensated prediction frame memory 12 as a reference image (stepST11). The loop filter part 11 can carry out the filtering process foreach coding block having the maximum size of the local decoded imagesignal inputted thereto or for each coding block of the local decodedimage signal. As an alternative, after the local decoded image signalscorresponding to all the macroblocks of one screen are inputted thereto,the loop filter part can carry out the filtering process on all themacroblocks of the one screen at a time.

The moving image encoding device repeatedly carries out the processes ofsteps ST4 to ST10 until the moving image encoding device completes theprocessing on each partition P_(i) ^(n) belonging to each of all thecoding blocks B^(n) into which the inputted image is divided by theblock dividing part 2 (step ST12). The variable length encoding part 13carries out a process of variable-length-encoding the compressed dataoutputted thereto from the transformation/quantization part 7, thecoding mode and the prediction difference encoding parameters which areoutputted thereto from the encoding controlling part 1, and the intraprediction parameter outputted thereto from the intra prediction part 4or the inter prediction parameter outputted thereto from themotion-compensated prediction part 5 to generate a bitstream into whichcoded data of the compressed data, coded data of the coding mode, codeddata of the prediction difference encoding parameters, and coded data ofthe intra prediction parameter or the inter prediction parameter aremultiplexed (step ST13).

Next, the process carried out by the intra prediction part 4 will beexplained concretely. FIG. 5 is an explanatory drawing showing anexample of the intra prediction parameter (intra prediction mode) whichcan be selected for each partition P_(i) ^(n) belonging to the codingblock B^(n). In the example of FIG. 5, prediction direction vectorscorresponding to intra prediction modes are shown, and it is designedthat a relative angle between prediction direction vectors becomes smallwith increase in the number of selectable intra prediction modes.

Hereafter, an intra process of generating an intra prediction signal ofthe luminance signal of a partition P_(i) ^(n) on the basis of the intraprediction parameter (intra prediction mode) for the luminance signalwill be explained. It is assumed that the size of the partition P_(i)^(n) is l_(i) ^(n)×m_(i) ^(n) pixels. FIG. 6 is an explanatory drawingshowing an example of pixels which are used when generating a predictedvalue of each pixel in the partition P_(i) ^(n) in the case of l_(i)^(n)=m_(i) ^(n)=4. Although the (2×l_(i) ^(n)+1) pixels in analready-encoded upper partition which is adjacent to the partition P_(i)^(n) and the (2×m_(i) ^(n)) pixels in an already-encoded left partitionwhich is adjacent to the partition P_(i) ^(n) are defined as the pixelsused for prediction in the example of FIG. 6, a smaller number of pixelsthan the pixels shown in FIG. 6 can be alternatively used forprediction. Although one row or column of pixels which are adjacent tothe partition P_(i) ^(n) are used for prediction in the example of FIG.6, two rows or columns of pixels or three or more rows or columns ofpixels can be alternatively used for prediction.

When an index value indicating the intra prediction mode for thepartition P_(i) ^(n) is 2 (average prediction), the intra predictionpart 4 generates a prediction image by using the average of the adjacentpixels in the upper partition and the adjacent pixels in the leftpartition as the predicted value of each pixel in the partition P_(i)^(n). When the index value indicating the intra prediction mode is otherthan 2 (average prediction), the intra prediction part generates thepredicted value of each pixel in the partition P_(i) ^(n) on the basisof a prediction direction vector v_(p)=(dx, dy) shown by the indexvalue. When relative coordinates in the partition P_(i) ^(n) (the upperleft pixel of the partition is defined as the point of origin) of eachpixel (prediction target pixel) for which the predicted value isgenerated are expressed as (x, y), the position of a reference pixelwhich is used for prediction is the point of intersection where thefollowing L and one of the lines of adjacent pixels intersect eachother.

$L = {\begin{pmatrix}x \\y\end{pmatrix} + {k\; \upsilon_{p}}}$

where k is a positive scalar value.

When a reference pixel is located at an integer pixel position, theinteger pixel is defined as the predicted value of the target pixel forprediction. In contrast, when a reference pixel is not located at aninteger pixel position, an interpolation pixel which is generated froman integer pixel adjacent to the reference pixel is defined as thepredicted value of the target pixel for prediction. In the example ofFIG. 6, because the reference pixel is not at an integer pixel position,the average of the values of the two pixels which are adjacent to thereference pixel is determined as the predicted value of the target pixelfor prediction. The intra prediction part can use not only the adjacenttwo pixels but also two or more adjacent pixels to generate aninterpolation pixel and determine the value of this interpolation pixelas the predicted value.

According to the same procedure, the intra prediction part 4 generatesprediction pixels for all the pixels of the luminance signal in thepartition P_(i) ^(n) and outputs an intra prediction image (P_(i) ^(n))generated thereby. The intra prediction parameter used for thegeneration of the intra prediction image (P_(i) ^(n)) are outputted tothe variable length encoding part 13 in order to multiplex the intraprediction parameter into the bitstream, as mentioned above. While theintra prediction image (P_(i) ^(n)) for the luminance signal in thepartition P_(i) ^(n) is generated in the above-mentioned way, an intraprediction image (P_(j) ^(n−1)) is generated for the luminance signal ina partition P_(j) ^(n−1) (j: the partition number of the higher layerpartition in the (n−1)th hierarchical layer to which the partition P_(i)^(n) belongs) in a similar manner, as shown in FIG. 7.

In this embodiment, as shown in FIG. 6, the intra prediction part isconstructed in such a way as to be able to select a portioncorresponding to the partition P_(i) ^(n), which is extracted from theintra prediction image (P_(i) ^(n−1)) for the higher layer partitionP_(j) ^(n−1), as a candidate for the intra prediction image (P_(i) ^(n))for the luminance signal in the partition P_(i) ^(n), in addition to theone that is generated from adjacent pixels of reference pixels adjacentto the partition P_(i) ^(n). More specifically, in a casein which ninedirections of intra prediction mode including an average prediction areprovided as the intra prediction modes, as shown in FIG. 9, nineprediction modes are further added when a higher layer partition exists.

Although an intra prediction is a means of predicting an unknown area inthe screen from a known area, the prediction efficiency can be improvedby selecting a candidate for the intra prediction image from predictionimages having different properties because the image signal has a largelocal change in a space direction. Therefore, in the case in which theintra prediction part is constructed as above in such a way that thenumber of directions in the intra prediction modes is increased, thedegree of similarity among prediction images generated is reduced ascompared with the case in which the number of intra prediction modes issimply increased, and hence a candidate for the intra prediction imagecan be selected from prediction images having many variations.Therefore, an increase in the number of directions in the intraprediction modes can increase the degree of improvement in the encodingefficiency as compared with an increase in the number of intraprediction modes. Further, because a prediction image for a lower layerpartition can be generated by extracting a part of a prediction imagefor a higher layer partition using the method in accordance with thepresent invention, an increase in the number of prediction directionsprovides an advantage of being able to reduce the amount of computationas compared with the case of an increase in the number of predictionmodes.

The intra prediction part also carries out an intra prediction processbased on the intra prediction parameter (intra prediction mode) on eachof the color difference signals of the partition P_(i) ^(n) according tothe same procedure as that according to which the intra prediction partcarries out the intra prediction process on the luminance signal, andoutputs the intra prediction parameter used for the generation of theintra prediction image to the variable length encoding part 13.

The variable length encoding part 13 variable-length-encodes the intraprediction parameter outputted thereto from the intra prediction part 4and multiplexes the codeword of the intra prediction parameter into thebitstream, as mentioned above. When encoding the intra predictionparameter, the variable length encoding unit 13 can select arepresentative prediction direction vector (prediction directionrepresentative vector) from among the prediction direction vectors of aplurality of directional predictions, express the intra predictionparameter by using an index indicating the prediction directionrepresentative vector (prediction direction representative index) and anindex indicating the difference between the prediction direction vectorand the prediction direction representative vector (prediction directiondifferential index), and carry out Huffman encoding, such as arithmeticencoding according to a probability model, for each of the indexes,thereby being able to reduce the code amount.

Next, the processing carried out by the moving image encoding deviceshown in FIG. 10 will be explained. When receiving the bitstreamgenerated by the moving image encoding device shown in FIG. 1, thevariable length decoding part 21 carries out a variable length decodingprocess on the bitstream to decode a frame size in units of a sequencewhich consists of one or more frames of pictures or in units of apicture (step ST21 of FIG. 11). After decoding the frame size, thevariable length decoding part 21 determines the maximum encoding blocksize determined by the moving image encoding device shown in FIG. 1 (themaximum size of each of the coding blocks which is a unit to beprocessed at a time when an intra prediction process or amotion-compensated prediction process is carried out), and the upperlimit on the hierarchy number (the number of hierarchical layers in thehierarchy in which each coding block having the maximum size ishierarchically divided into blocks) according to the same procedure asthat which the moving image encoding device uses (step ST22).

For example, when the maximum size of each of the coding blocks isdetermined for all the pictures according to the resolution of theinputted image in the moving image encoding device, the variable lengthdecoding part determines the maximum size of each of the coding blockson the basis of the frame size which the variable length decoding parthas decoded previously according to the same procedure as that which themoving image encoding device shown in FIG. 1 uses. When both the maximumsize of each of the coding blocks and the number of hierarchical layersof the coding blocks are multiplexed into the bitstream by the movingimage encoding device, the variable length decoding part decodes thebitstream to acquire both the maximum size of each of the coding blocksand the number of hierarchical layers of the coding blocks.

After determining both the maximum size of each of the coding blocks andthe number of hierarchical layers of the coding blocks, the variablelength decoding part 21 specifies the coded data associated with eachcoding block from the coded data multiplexed into the bitstream bygrasping the state of the hierarchical division of the inputted imageinto the coding blocks with each coding block having the maximum sizebeing set as a starting point, and decodes the coded data to acquire thecoding mode assigned to each of the coding blocks. The variable lengthdecoding part 21 then refers to the division information about thedivision into partitions P_(i) ^(n) belonging to the coding block B^(n)included in the coding mode to specify the coded data associated witheach of the partitions P_(i) ^(n) from the coded data multiplexed intothe bitstream (step ST23). The variable length decoding part 21variable-length-decodes the coded data associated with each of thepartitions P_(i) ^(n) to acquire the compressed data, the predictiondifference encoding parameters, and the intra prediction parameter orthe inter prediction parameter, and outputs the compressed data and theprediction difference encoding parameters to the inversequantization/inverse transformation part 25 and also outputs the codingmode and the intra prediction parameter or the inter predictionparameter to the selection switch 22 (step ST24).

For example, when the prediction direction representative index and theprediction direction differential index are multiplexed into thebitstream, the variable length decoding part entropy-decodes theprediction direction representative index and the prediction directiondifferential index by using, for example, arithmetic decoding methodsaccording to their respective probability models, respectively, tospecify the intra prediction parameter from the prediction directionrepresentative index and the prediction direction differential index. Asa result, even when the code amount of the intra prediction parameter isreduced in the moving image encoding device, the moving image decodingdevice can decode the intra prediction parameter correctly.

When the coding mode of each partition P_(i) ^(n) belonging to thecoding block B^(n) outputted thereto from the variable length decodingpart 21 is an intra coding mode, the selection switch 22 outputs theintra prediction parameter outputted thereto from the variable lengthdecoding part 21 to the intra prediction part 23, whereas when thecoding mode is an inter coding mode, the selection switch outputs theinter prediction parameter outputted thereto from the variable lengthdecoding part 21 to the motion compensation part 24.

When receiving the intra prediction parameter from the selection switch22 (step ST25), the intra prediction part 23 carries out an intraprediction process on each partition P_(i) ^(n) to generate an intraprediction image (P_(i) ^(n)) on the basis of the intra predictionparameter (step ST26), like the intra prediction part 4 shown in FIG. 1.More specifically, the intra prediction part 23 carries out anintra-frame prediction process on each partition P_(i) ^(n) to generatea prediction image (P_(i) ^(n)) by using already-decoded pixels whichare stored in the memory 27 for intra prediction and which are adjacentto the partition P_(i) ^(n) or already-decoded pixels which are adjacentto a higher layer partition P_(i) ^(n−1) to which the partition P_(i)^(n) belongs on the basis of the intra prediction parameter.

When receiving the inter prediction parameter from the selection switch22 (step ST25), the motion compensation part 24 carries out an interprediction process on each partition P_(i) ^(n) to generate an interprediction image (P_(i) ^(n)) on the basis of the inter predictionparameter, like the motion-compensated prediction part 5 shown in FIG. 1(step ST27). More specifically, the motion compensation part 24 carriesout a motion-compensated prediction process on each partition P_(i) ^(n)to generate an inter prediction image (P_(i) ^(n)) by using one or moreframes of reference images stored in the motion-compensated predictionframe memory 29 on the basis of the inter prediction parameter.

When receiving the prediction difference encoding parameters from thevariable length decoding part 21, the inverse quantization/inversetransformation part 25 inverse-quantizes the compressed data associatedwith the coding block, which are outputted thereto from the variablelength decoding part 21, by using the quantization parameter included inthe prediction difference encoding parameters, and carries out aninverse transformation process (e.g., an inverse DCT (inverse discretecosine transform) or an inverse transformation process such as aninverse KL transform) on the compressed data inverse-quantized therebyin units of a block having the transformation block size included in theprediction difference encoding parameters, and outputs the compresseddata on which the inverse quantization/inverse transformation part hascarried out the inverse transformation process to the adding part 26 asa decoded prediction difference signal (signal showing a pre-compresseddifference image) (step ST28).

The adding part 26 generates a decoded image signal showing a decodedpartition image or a decoded image which is a group of decoded partitionimages by adding the decoded prediction difference signal and theprediction signal showing the prediction image (P_(i) ^(n)) generated bythe intra prediction part 23 or the motion-compensated prediction part24, and outputs the decoded image signal to the loop filter part 28(step ST29). Further, this decoded image is stored in the memory 27 forintra prediction in order that the decoded image can be used for intraprediction.

When receiving the decoded image signal from the adding part 26, theloop filter part 28 compensates for an encoding distortion included inthe decoded image signal, and stores the decoded image shown by thedecoded image signal on which the loop filter part has carried out theencoding distortion compensation in the motion-compensated predictionframe memory 29 as a reference image and also outputs the decoded imageas a reproduced image (step ST30). The loop filter part 28 can carry outthe filtering process for each coding block having the maximum size ofthe decoded image signal inputted thereto or each coding block of thedecoded image signal. As an alternative, after the decoded image signalscorresponding to all the macroblocks of one screen are inputted, theloop filter part can carry out the filtering process on all themacroblocks of the one screen at a time. The processes of steps ST23 toST29 are repeatedly carried out until the processing on each partitionP_(i) ^(n) belonging to each of all the coding blocks B^(n) is completed(step ST31).

As can be seen from the above description, because the intra predictionpart 4 of the moving image encoding device in accordance with thisEmbodiment 1 is constructed in such a way as to, when the coding modeselected by the encoding controlling unit 1 is an intra prediction mode,carryout an intra-frame prediction process using pixels adjacent to apartition P_(i) ^(n) which is generated through the division by theblock dividing unit 2 or pixels adjacent to a higher layer partitionP_(j) ^(n−1) to which the partition P_(i) ^(n) belongs to generate aprediction image (P_(i) ^(n)), there is provided an advantage of beingable to increase the degree of improvement in the encoding efficiencywith a small amount of computation.

Further, because the intra prediction part 23 of the moving imagedecoding device in accordance with this Embodiment 1 is constructed insuch a way as to, when the coding mode variable-length-decoded by thevariable length decoding part 21 is an intra prediction mode, carry outan intra-frame prediction process using already-decoded pixels which arestored in the memory 27 for intra prediction and which are adjacent to apartition P_(i) ^(n) or already-decoded pixels adjacent to a higherlayer partition P_(j) ^(n−1) to which the partition P_(i) ^(n) belongsto generate a prediction image (P_(i) ^(n)), there is provided anadvantage of being able to correctly decode the coded data in which animprovement in the encoding efficiency is provided to acquire the movingimage.

Embodiment 2

Although the example in which the variable length encoding part 13 ofthe moving image encoding device variable-length-encodes the intraprediction parameter for a partition to be encoded is shown inabove-mentioned Embodiment 1, the variable length encoding part canvariable-length-encode an intra merge flag indicating whether or not theintra prediction parameter for the partition to be encoded is identicalto that for a partition adjacent to the partition, and can alsovariable-length-encode an intra merge direction for specifying theadjacent partition when the intra prediction parameter for the partitionto be encoded is identical to that for the adjacent partition, otherwisevariable-length-encode the intra prediction parameter for the partitionto be encoded.

Further, although the example in which the variable length decoding part21 of the moving image decoding device variable-length-decodes the codeddata associated with a partition to be decoded to acquire the intraprediction parameter for the partition is shown in above-mentionedEmbodiment 1, the variable length decoding part canvariable-length-decode the coded data associated with the partition tobe decoded to acquire the intra merge flag indicating whether or not theintra prediction parameter for the partition to be decoded is identicalto that for a partition adjacent to the partition, and can alsovariable-length-decode the coded data to acquire the intra mergedirection for specifying the adjacent partition when the intraprediction parameter for the partition to be decoded is identical tothat for the adjacent partition, otherwise variable-length-decode thecoded data to acquire the intra prediction parameter for the partitionto be encoded.

Because a moving image encoding device and a moving image decodingdevice in accordance with this embodiment have the same structures asthose according to above-mentioned Embodiment 1 except for componentscorresponding to encoding and decoding of an intra prediction parameter,only the encoding and decoding of the intra prediction parameter will beexplained in this Embodiment 2.

When variable-length-encoding the intra prediction parameter for apartition P_(i) ^(n) to be encoded (target to be processed), a variablelength encoding part 13 of the moving image encoding device inaccordance with this Embodiment 2 variable-length-encodes an intra mergeflag indicating whether the intra prediction parameter is identical tothat for a partition adjacent to the partition P_(i) ^(n), as shown inFIG. 12(A). When the intra merge flag indicates that the intraprediction parameter is identical to that for an adjacent partition, thevariable length encoding part 13 also variable-length-encodes an intramerge direction showing which one of adjacent partitions has the sameintra prediction parameter as the partition to be encoded. In contrast,when the intra merge flag indicates that the intra prediction parameteris not identical to that for any adjacent partition, the variable lengthencoding part variable-length-encodes the intra prediction parameter forthe partition P_(i) ^(n) to be encoded (in this case, the variablelength encoding part carries out the same encoding as that shown inabove-mentioned Embodiment 1), as shown in FIG. 12(B).

For example, when a partition adjacent to the left of the partition tobe encoded and a partition adjacent to the top of the partition to beencoded are listed as adjacent partition candidates, as shown in FIG.13(A), a flag indicating whether the intra prediction parameter for thepartition to be encoded is identical to that for the partition adjacentto the left of the partition to be encoded or those for the partitionadjacent to the top of the partition to be encoded is provided as theintra merge direction. As an alternative, a partition adjacent to theleft of the partition to be encoded, a partition adjacent to the topleft of the partition to be encoded, and a partition adjacent to the topof the partition to be encoded are listed can be listed as adjacentpartition candidates, and a flag indicating which one of the threecandidates has the same intra prediction parameter as the partition tobe encoded can be provided as the intra merge direction, as shown inFIG. 13(B). In a case in which all adjacent partitions each of which canbe the target for flag encoding have the same intra predictionparameter, it is not necessary to encode the intra merge direction as amatter of course. Therefore, the variable length encoding part can beconstructed in such a way as not to encode the intra merge direction inthis case, as shown in FIG. 12(D). Further, the variable length encodingpart can be constructed in such a way as to, when there are two or morepartitions adjacent to the top or left of the partition to be encoded,scan the partitions in a direction away from a top left side of thepartition to be encoded, for example, and select the first partitionwhich has been encoded in an intra prediction mode as the adjacentpartition to the top or left of the partition to be encoded.

Further, when the number NumN of selectable intra prediction directionsin the adjacent partition is not equal to the number NumC of selectableintra prediction directions in the partition to be encoded, the variablelength encoding part can be constructed in the following way. Morespecifically, when NumN<NumC, the variable length encoding part brings arepresentative prediction direction vector, among the predictiondirection vectors of a plurality of directional predictions in thepartition P_(i) ^(n) to be encoded, into correspondence with one of theselectable intra prediction directions in the adjacent partition todetermine whether the intra prediction parameter for the partition P_(i)^(n)to be encoded matches that for the adjacent partition. In this case,when determining that the intra prediction parameter for the partitionP_(i) ^(n) to be encoded match those for the adjacent partition, thevariable length encoding part encodes an intra prediction directionresidual parameter showing which one of the plurality of correspondingintra prediction directions has been selected for the partition P_(i)^(n) to be encoded, as shown in FIG. 12(C).

In contrast, when NumN>NumC, the variable length encoding part brings arepresentative prediction direction vector, among the predictiondirection vectors of a plurality of directional predictions in theadjacent partition, into correspondence with one of the selectable intraprediction directions in the partition P_(i) ^(n) to be encoded todetermine whether the intra prediction parameter for the partition P_(i)^(n) to be encoded matches that for the adjacent partition. In thiscase, the variable length encoding part does not have to encode theintra prediction direction residual parameter. Because it can beconsidered that an intra prediction mode has a direction depending onthe texture of the image to be encoded, a similar prediction mode easilyoccurs locally. Therefore, the variable length encoding part can encodethe intra prediction parameter with a smaller amount of information byencoding the intra prediction parameter by using the intra merge flagand the intra merge direction.

When variable-length-decoding the intra prediction parameter in apartition P_(i) ^(n) to be decoded (target to be processed), a variablelength decoding part 21 of the moving image decoding device inaccordance with this Embodiment 2 variable-length-decodes the intramerge flag indicating whether or not the intra prediction parameter isidentical to that for a partition adjacent to the partition P_(i) ^(n),as shown in FIG. 12(A). Further, when the intra merge flag indicatesthat the intra prediction parameter is identical to that for an adjacentpartition, the variable length decoding part 21 variable-length-decodesthe intra merge direction showing which one of adjacent partitions hasthe same intra prediction parameter as the partition to be decoded. Incontrast, when the intra merge flag indicates that the intra predictionparameter is not identical to that for an adjacent partition, thevariable length decoding part variable-length-decodes the intraprediction parameter in the partition P_(i) ^(n) to be decoded (in thiscase, the variable length decoding part carries out the same decoding asthat shown in above-mentioned Embodiment 1), as shown in FIG. 12(B).

For example, when a partition adjacent to the left of the partition tobe decoded and a partition adjacent to the top of the partition to bedecoded are listed as adjacent partition candidates, as shown in FIG.13(A), a flag indicating whether the intra prediction parameter for thepartition to be decoded is identical to that for the partition adjacentto the left of the partition to be decoded or those for the partitionadjacent to the top of the partition to be decoded is provided as theintra merge direction. As an alternative, a partition adjacent to theleft of the partition to be decoded, a partition adjacent to the topleft of the partition to be decoded, and a partition adjacent to the topof the partition to be decoded can be listed as adjacent partitioncandidates, and a flag indicating which one of the three candidates hasthe same intra prediction parameter as the partition to be decoded canbe provided as the intra merge direction, as shown in FIG. 13(B). In acase in which all adjacent partitions each of which can be the targetfor flag encoding have the same intra prediction parameter, it is notnecessary to encode the intra merge direction as a matter of course.Therefore, the variable length decoding part can be constructed in sucha way as not to decode the intra merge direction in this case, as shownin FIG. 12(D). Further, the variable length decoding part can beconstructed in such a way as to, when there are two or more partitionsadjacent to the top or left of the partition to be decoded, scan thepartitions in a direction away from a top left side of the partition tobe decoded, for example, and select the first partition which has beenencoded in an intra prediction mode as the adjacent partition to the topor left of the partition to be decoded.

Further, when the number NumN of selectable intra prediction directionsin the adjacent partition is not equal to the number NumC of selectableintra prediction directions in the partition to be decoded, the variablelength decoding part can be constructed in the following way. Morespecifically, when NumN<NumC, the variable length decoding part brings arepresentative prediction direction vector, among the predictiondirection vectors of a plurality of directional predictions in thepartition P_(i) ^(n) to be decoded, into correspondence with one of theselectable intra prediction directions in the adjacent partition todetermine whether the intra prediction parameter for the partition P_(i)^(n)to be decoded matches that for the adjacent partition. In this case,when determining that the intra prediction parameter for the partitionP_(i) ^(n) to be decoded matches that for the adjacent partition, thevariable length decoding part encodes an intra prediction directionresidual parameter showing which one of the plurality of correspondingintra prediction directions has been selected for the partition P_(i)^(n) to be decoded, as shown in FIG. 12(C).

In contrast, when NumN>NumC, the variable length decoding part brings arepresentative prediction direction vector, among the predictiondirection vectors of a plurality of directional predictions in theadjacent partition, into correspondence with one of the selectable intraprediction directions in the partition P_(i) ^(n) to be decoded todetermine whether the intra prediction parameter for the partition P_(i)^(n) to be decoded matches that for the adjacent partition. In thiscase, the variable length decoding part does not have to decode theintra prediction direction residual parameter. Because the variablelength decoding part is constructed in this way, the variable lengthdecoding part can properly decode the intra prediction parameter whichis encoded by the moving image encoding device in accordance with thisEmbodiment 2. As a matter of course, the variable length encoding unitand the variable length decoding unit in accordance with this Embodiment2 can also be applied to a case in which the partition to be processedand the adjacent partition are not placed in a prediction mode, asexplained in this Embodiment 1, in which part of an intra predictionimage in a higher hierarchical layer is extracted.

While the invention has been described in its preferred embodiments, itis to be understood that an arbitrary combination of two or more of theabove-mentioned embodiments can be made, various changes can be made inan arbitrary component according to any one of the above-mentionedembodiments, and an arbitrary component according to any one of theabove-mentioned embodiments can be omitted within the scope of theinvention.

INDUSTRIAL APPLICABILITY

As mentioned above, because the image coding device, the image decodingdevice, the image coding method, and the image decoding method inaccordance with the present invention are capable of increasing thedegree of improvement in the encoding efficiency which is caused by anincrease in the number of directional predictions with a small amount ofcomputation and a small code amount, the image coding device and theimage coding method are suitable for use as an image coding device forand an image coding method of encoding a moving image with a high degreeof efficiency, and so on, and the image decoding device and the imagedecoding method are suitable for use as an image decoding device for andan image decoding method of decoding an encoded moving image with a highdegree of efficiency, and so on.

1 encoding controlling part (encoding controlling unit), 2 blockdividing part (block dividing unit), 3 select switch (prediction imagegenerating unit), 4 intra prediction part (prediction image generatingunit), 5 motion-compensated prediction part (prediction image generatingunit), 6 subtracting part (difference image generating unit), 7transformation/quantization part (image compression unit), 8 inversequantization/inverse transformation part, 9 adding part, memory forintra prediction, 11 loop filter part, 12 motion-compensated predictionframe memory, 13 variable length encoding part (variable length encodingunit), 21 variable length decoding part (variable length decoding unit),22 select switch (prediction image generating unit), 23 intra predictionpart (prediction image generating unit), 24 motion compensation part(prediction image generating unit), 25 inverse quantization/inversetransformation part (difference image generating unit), 26 adding part(decoded image generating unit), 27 memory for intra prediction, 28 loopfilter part, 29 motion-compensated prediction frame memory.

1. An image decoding device comprising: a decoder for performing adecoding process on a coded data multiplexed into a bitstream to obtaina coding mode and a compressed data for each of coding blocks divided upto an upper limit on a number of hierarchical layers; an intraprediction unit for carrying out an intra prediction process on apartition of the coding block to obtain an intra prediction image whenan intra prediction mode is selected based on the coding mode; adifference image generator to carry out an inverse transformationprocess on the compressed data to obtain a prediction difference signal;wherein said decoder obtains from said bitstream an intra merge flagindicating whether or not an intra prediction parameter of said currentpartition is identical to that of an adjacent partition located above orto the left of said current partition, wherein when there are two ormore partitions adjacent to top or left of said current partition, afirst partition in a direction away from a top left of said currentpartition is selected as said adjacent partition, wherein when saidintra merge flag indicates that said intra prediction parameter of saidcurrent partition is identical to that of said adjacent partition, saiddecoder obtains from said bitstream an intra merge direction specifying,out of said adjacent partitions located above and to the left of saidcurrent partition, the adjacent partition whose intra predictionparameter is identical to that of said current partition, and when saidintra prediction parameter of said current partition is not identical tothat of said adjacent partition, said decoder obtains from saidbitstream said intra prediction parameter for said current partition. 2.An image coding device comprising: an encoding control unit foridentifying a coding mode of each of coding blocks which is obtained byhierarchically dividing a coding block having a maximum size which isunit for coding process an intra prediction unit for carrying out anintra prediction process on a partition of the coding block to obtainthe intra prediction image when an intra prediction mode is selectedbased on the coding mode; a compressor to carry out a transformationprocess on a difference image generated from the coding block and theintra prediction image to obtain a compressed data of the differenceimage, a coding unit for performing a coding on the coding mode, anintra prediction parameter used to generate the intra prediction image,and the compressed data obtained by said compressor, and said codingunit multiplexing a coded data of said coding mode, said intraprediction parameter, and said compressed data to a bitstream; saidcoding unit performing a coding on an intra merge flag indicatingwhether or not said intra prediction parameter of said current partitionis identical to an intra prediction parameter of an adjacent partitionlocated above or to the left of said intra predicted partition, andmultiplexing said intra merge flag into said bitstream; wherein whenthere are two or more partitions adjacent to top or left of said currentpartition, a first partition in a direction away from a top left of saidcurrent partition is selected as said adjacent partition, wherein whensaid intra prediction parameter of said current partition is identicalto that of said adjacent partition, said coding unit performs a codingon an intra merge direction specifying, out of adjacent partitionslocated above and to the left of said current partition, the adjacentpartition whose intra prediction parameter is identical to that of saidcurrent partition, and said intra merge direction is multiplexed intosaid bitstream, and when said intra prediction parameter of said currentpartition is not identical to that of said adjacent partition, saidcoding unit performs a coding on said intra prediction parameter of saidcurrent partition, and said intra prediction parameter is multiplexedinto said bitstream.
 3. A non-transitory computer-readable mediumstoring a bit stream having computer-executable coded data specifyingcoding mode of a hierarchically divided coding block, said coded dataand a maximum size of the coding block being multiplexed into said bitstream, said coded data comprising: an intra merge flag indicatingwhether or not an intra prediction parameter of a current partition isidentical to that of an adjacent partition located above or to the leftof said current partition; an intra merge direction for said currentpartition of which intra prediction parameter is identical to that ofsaid adjacent partition, said intra merge direction specifying, out ofadjacent partitions located above and to the left of said currentpartition, the adjacent partition whose intra prediction parameter isidentical to that of said current partition; and an intra predictionparameter for said current partition of which intra prediction parameteris not identical to that of said adjacent partition, said intraprediction parameter used when said intra merge flag indicates that anintra prediction parameter of said partition is not identical to that ofsaid adjacent partition; wherein when there are two or more partitionsadjacent to top or left of said current partition, a first partition ina direction away from a top left of said current partition is selectedas said adjacent partition.
 4. An image decoding method comprising:performing a decoding process on a coded data multiplexed into abitstream to obtain a coding mode and a compressed data for each ofcoding blocks divided up to an upper limit on a number of hierarchicallayers; carrying out an intra prediction process on a partition of thecoding block to obtain an intra prediction image when an intraprediction mode is selected based on the coding mode; carrying out aninverse transformation process on the compressed data to obtain aprediction difference signal; wherein said method obtains from saidbitstream an intra merge flag indicating whether or not an intraprediction parameter of said current partition is identical to that ofan adjacent partition located above or to the left of said currentpartition, wherein when there are two or more partitions adjacent to topor left of said current partition, a first partition in a direction awayfrom a top left of said current partition is selected as said adjacentpartition, wherein when said intra merge flag indicates that said intraprediction parameter of said current partition is identical to that ofsaid adjacent partition, said method obtains from said bitstream anintra merge direction specifying, out of said adjacent partitionslocated above and to the left of said current partition, the adjacentpartition whose intra prediction parameter is identical to that of saidcurrent partition, and when said intra prediction parameter of saidcurrent partition is not identical to that of said adjacent partition,said method obtains from said bitstream said intra prediction parameterfor said current partition.
 5. An image coding method comprising:identifying a coding mode of each of coding blocks which is obtained byhierarchically dividing a coding block having a maximum size which isunit for coding process carrying out an intra prediction process on apartition of the coding block to obtain the intra prediction image whenan intra prediction mode is selected based on the coding mode; carryingout a transformation process on a difference image generated from thecoding block and the intra prediction image to obtain a compressed dataof the difference image, performing a coding on the coding mode, anintra prediction parameter used to generate the intra prediction image-,and the compressed data obtained, and said method multiplexing a codeddata of said coding mode, said intra prediction parameter, and saidcompressed data to a bitstream; said method performing a coding on anintra merge flag indicating whether or not said intra predictionparameter of said current partition is identical to an intra predictionparameter of an adjacent partition located above or to the left of saidintra predicted partition, and multiplexing said intra merge flag intosaid bitstream; wherein when there are two or more partitions adjacentto top or left of said current partition, a first partition in adirection away from a top left of said current partition is selected assaid adjacent partition, wherein when said intra prediction parameter ofsaid current partition is identical to that of said adjacent partition,said method performs a coding on an intra merge direction specifying,out of adjacent partitions located above and to the left of said currentpartition, the adjacent partition whose intra prediction parameter isidentical to that of said current partition, and said intra mergedirection is multiplexed into said bitstream, and when said intraprediction parameter of said current partition is not identical to thatof said adjacent partition, said method performs a coding on said intraprediction parameter of said current partition, and said intraprediction parameter is multiplexed into said bitstream.